A Step towards a Benchmark Repository for E-commerce

A benchmark is used to measure and compare the performance of like systems. It is also used to estimate the scalability of a system in terms of the number of users and/or transactions that a system can support and system response times under different loads and hardware/software deployment platforms. Because of the multidimensional aspects of e-commerce and the various emerging and distinct e-commerce business models, one single benchmark is not suitable for the purposes of examining scalability and determining throughput and response time of an e-commerce application under consideration. The diverse needs of small-to-medium enterprises (SMEs) and big business provide additional motivation for a benchmark suite for e-commerce. This paper advocates development of a suite of benchmarks of distinguished business models that represent existing and emerging e-commerce applications in terms of their distinct features. In particular, the paper develops a benchmark for one such business model, a model that is based on a cybermediary or electronic broker, e-broker. Its distinguishing feature is the buy transaction consisting of a set of nested, distributed transactions that perform credit checks, user authentication, and product availability searches. Two implementations of the benchmark specification are described: one based on Microsoft's COM technology while the other one is based on CORBA technology. Sample benchmark results are also presented and discussed in order to demonstrate the usefulness of the benchmark when examining usage of distinct technologies in application implementation and also when examining the scalability of the application.